markup: add --citeproc to pandoc converter #9953
Conversation
|
|
|
@shoeffner thanks for this. This PR is certainly easier to reason about (because of its size) than some of the others I've seen. That said, the test failure on Linux shows that this needs some rethinking, as that flag is not available in the version we test on, fetched via: I assume this flag was added recently? I'm not sure how to handle this. |
|
The Linux build seems to use pandoc 2.5, but --citeproc is only supported > 2.11 (https://pandoc.org/releases.html#pandoc-2.11-2020-10-11 – on macOS, 2.18 is installed, so I didn't notice) – before If I remember correctly, pandoc-citeproc had to be installed manually, but I will check it out in a container and see what I can do! |
|
choco and brew seem to install 2.18, so Windows and macOS should not need any specific handling. On Linux we have everything from 2.5 (Ubuntu 20.04, e.g. GH actions) to 2.17 (Arch Linux); the most common version seems to be 2.9 (Debian stable, Ubuntu 22.04, and other distros), here This makes the pull request much more complicated than what I was aiming for, so feel free to close it again. My current approach is now: If the citations are not supported (due to the version or pandoc-citeproc dependency), SupportsCitations is false and the corresponding tests are skipped. I also added a remark to the docs. An alternative could be to check for the version and ignore pandoc-citeproc. This would make it much simpler and whoever wants to use citeproc might also want to use other pandoc features anyways. But it puts some burden on the users who have to install a newer pandoc version. What do you think? |
shoeffner
changed the title
shoeffner
force-pushed
the
citeproc
branch
3 times, most recently
from
c71d9f8
to
c000c6c
Compare
|
I rebased the commits and squashed them. |
shoeffner
changed the title
shoeffner
changed the title
shoeffner
changed the title
|
I updated the PR again, I made a mistake ( |
|
Rebased on master. |
shoeffner
force-pushed
the
citeproc
branch
2 times, most recently
from
7dfa143
to
2f5412d
Compare
|
Sorry for the late reaction. I fixed it to use |
|
Rebased on master. |
|
Okay, comparing strings 2.5 and 2.11 was a stupid idea... Now it has a proper type and a function to compare two instances. 3c5e31f#diff-f662bf93836a7230b77cdb532095a04622eb9eda3054124e8ba9399786870efaR88-R94 Additionally, I included some test cases for those comparisons. I hope it will work now ;-) |
|
Unsupported pandoc should now cause the tests to SKIP instead of causing a panic. |
|
Rebased on the current tip of the master branch. |
|
@shoeffner thanks for putting this PR together. It is quite useful. Do you have a sense if this, or something like it will end up getting merged? Being able to put citations in text is quite important. |
Adds the citeproc filter to the pandoc converter. There are several PRs for it this feature already. However, I think simply adding `--citeproc` is the cleanest way to enable this feature, with the option to flesh it out later, e.g., in gohugoio#7529. Some PRs and issues attempt adding more config options to Hugo which indirectly configure pandoc, but I think simply configuring Pandoc via Pandoc itself is simpler, as it is already possible with two YAML blocks -- one for Hugo, and one for Pandoc: --- title: This is the Hugo YAML block --- --- bibliography: assets/pandoc-yaml-block-bibliography.bib ... Document content with @citation! There are other useful options, e.g., gohugoio#4800 attempts to use `nocite`, which works out of the box with this PR: --- title: This is the Hugo YAML block --- --- bibliography: assets/pandoc-yaml-block-bibliography.bib nocite: | @* ... Document content with no citations but a full bibliography: ## Bibliography Other useful options are `csl: ...` and `link-citations: true`, which set the path to a custom CSL file and create HTML links between the references and the bibliography. The following issues and PRs are related: - Add support for parsing citations and Jupyter notebooks via Pandoc and/or Goldmark extension gohugoio#6101 Bundles multiple requests, this PR tackles citation parsing. - WIP: Bibliography with Pandoc gohugoio#4800 Passes the frontmatter to Pandoc and still uses `--filter pandoc-citeproc` instead of `--citeproc`. - Allow configuring Pandoc gohugoio#7529 That PR is much more extensive and might eventually supersede this PR, but I think --bibliography and --citeproc should be independent options (--bibliography should be optional and citeproc can always be specified). - Pandoc - allow citeproc extension to be invoked, with bibliography. gohugoio#8610 Similar to gohugoio#7529, gohugoio#8610 adds a new config option to Hugo. I think passing --citeproc and letting the users decide on the metadata they want to pass to pandoc is better, albeit uglier.
|
Thanks for getting back to this, I rebased the changes on the master. I don't know whether this will be merged and if the pipelines still work, nor about the progress on alternative approaches, as I haven't followed the current developments much. Maybe @bep knows more about it. |
Adds the citeproc filter to the pandoc converter.
There are several PRs for it this feature already. However, I think
simply adding
--citeprocis the cleanest way to enable this feature,with the option to flesh it out later, e.g., in #7529.
Some PRs and issues attempt adding more config options to Hugo which
indirectly configure pandoc, but I think simply configuring Pandoc via
Pandoc itself is simpler, as it is already possible with two YAML
blocks -- one for Hugo, and one for Pandoc:
There are other useful options, e.g., #4800 attempts to use
nocite,which works out of the box with this PR:
Other useful options are
csl: ...andlink-citations: true, whichset the path to a custom CSL file and create HTML links between the
references and the bibliography.
The following issues and PRs are related:
Add support for parsing citations and Jupyter notebooks via Pandoc and/or Goldmark extension Add support for parsing citations and Jupyter notebooks via Pandoc and/or Goldmark extension #6101
Bundles multiple requests, this PR tackles citation parsing.
WIP: Bibliography with Pandoc WIP: Bibliography with Pandoc #4800
Passes the frontmatter to Pandoc and still uses
--filter pandoc-citeprocinstead of--citeproc.Allow configuring Pandoc Allow configuring Pandoc #7529
That PR is much more extensive and might eventually supersede this PR,
but I think --bibliography and --citeproc should be independent
options (--bibliography should be optional and citeproc can always be
specified).
Pandoc - allow citeproc extension to be invoked, with bibliography. Pandoc - allow citeproc extension to be invoked, with bibliography. #8610
Similar to Allow configuring Pandoc #7529, Pandoc - allow citeproc extension to be invoked, with bibliography. #8610 adds a new config option to Hugo.
I think passing --citeproc and letting the users decide on the
metadata they want to pass to pandoc is better, albeit uglier.
Note:
This PR also adds a tiny little bit of unrelated documentation to the external helpers by pointing out MathJax superficially. This certainly needs improvements, but is out of the scope for this PR.